Easier comparison of bets in evaluation does not reduce classical preference reversals: Evidence against a context-dependent explanation.

In preference reversals, subjects express different rankings over a set of alternatives depending on how preferences are elicited. In classical reversal tasks, for instance, subjects often select a safe bet over a risky one when given a choice between the two in a pair, but then assign a higher monetary evaluation to the risky bet. Motivated by a rich literature on context-dependent preferences, we conjecture that comparisons across bets in a pair can influence both Choice and Evaluation. Yet deciders are less likely to mentally compare the bets in the latter case, as bets are typically evaluated in isolation. This asymmetry between Choice and Evaluation is, we surmise, one cause of the reversals. If we further assume that memory decay affects mental comparisons in Evaluation, the account predicts order and timing effects on the reversal probability. We run several treatments designed to facilitate or hinder the retrieval from memory of the alternative bet during evaluation of a bet. However, the reversal rate does not vary across treatments in the predicted direction, and we find no systematic order or timing effects. We conclude that reversals are not influenced by the ease with which subjects recall the alternative bet during the evaluations, which suggests in turn that a relatively smaller frequency of comparisons across bets during the (typically isolated) evaluations is not a significant cause of reversals.

Esteemed Colleagues: A Model of the Effect of Open Data on Selective Reporting of Scientific Results.

Open data, the practice of making available to the research community the underlying data and analysis codes used to generate scientific results, facilitates verification of published results, and should thereby reduce the expected benefit (and hence the incidence) of p-hacking and other forms of academic dishonesty. This paper presents a simple signaling model of how this might work in the presence of two kinds of cost. First, reducing the cost of "checking the math" increases verification and reduces falsification. Cases where the author can choose a high or low verification-cost regime (that is, open or closed data) result in unraveling; not all authors choose the low-cost route, but the best do. The second kind of cost is the cost to authors of preparing open data. Introducing these costs results in that high- and low-quality results being published in both open and closed data regimes, but even when the costs are independent of research quality open data is favored by high-quality results in equilibrium. A final contribution of the model is a measure of "science welfare" that calculates the ex-post distortion of equilibrium beliefs about the quality of published results, and shows that open data will always improve the aggregate state of knowledge.